NHS-R Webinar - Using a Raspberry pi cluster to host a Shiny app.

Pablo Leon-Rodenas |>
pablo.leonrodenas@nhs.net
Senior Analytics Developer | Analytical Products team

19 Jan 2022

Shiny on RaspberryPi cluster - [S01-03]

Aim This presentation describes
the steps required to setup a Raspberry Pi cluster and preparations needed to
turn it into a Shiny Server.

This is an small experiment I designed at home following several resources available online. Basically I wanted to explore how to access the cluster of Raspbery Pi's and also how to turn them into a Shiny Server.

List of topics Today we will cover these four topics:

  • 1. Different ways of connecting remotely to the Raspberry Pi
  • 2. How to setup a Raspberry pi cluster,
  • 3. Generate a SSH key on Windows to log into each Raspberry Pi's in the Cluster
  • 4. Briefly description of the process to create a Shiny server (following R Studio Community blog posts)

Shiny app created using standard laptop

Example of a Shiny dashbaord I build on my laptop

alt text

alt text alt text

Main resource used as inspiration for this presentation

For this experimental project I followed the post below from Andres Castro Socolich from the R studio community https://community.rstudio.com/t/setting-up-your-own-shiny-server-rstudio-server-on-a-raspberry-pi-3b/18982

alt text

Hardware and cluster components setup

This is the cluster I put for this experiment, upgraded also up to five Raspberry Pies

For this experimental project I followed the post below from Andres Castro Socolich from the R studio community

How to access remotely your Raspberry pi

To build the Raspberry pi cluster, we can choose to access it in two different ways

1. Via SSH -This is the common way of accessing several Raspberr pies from a single computer
-When flashing the Rasbpian bulleye OS into the SD card, press 'CTRL+SHIFT+x' to: -Setup wifi -Enable SSH -Setup new password

2.Via a remote desktop
-We will use VNC and install TIGHTVNCSERVER client in our laptop and the Raspberry pi cluster to enable communication via remote desktop among them. https://www.realvnc.com/en/connect/download/vnc/ Terminal commands on Raspberry Pi

sudo apt-get install tightvncserver

sudo apt-get install xrdp

- It is important to Enable graphical remote access to achieve this in the config menu

sudo raspi-config

Troubleshooting when connecting via SSH

It is possible that we encounter the following error below when trying to access our Raspberry Pi via SSH. We need to remove the keys belonging to that specific Raspberry Pi that we want to access using the command below.

c:\Users\PABLO>ssh-keygen -R 192.168.0.11

alt text

We need to remove the existing keys from the IP registry to clear the SSH connection

Public SSH setup for our Raspberry pi cluster

Once several Raspberry pi 4 are connected to our local network, we create a public SSH to share across each node.

https://pimylifeup.com/raspberry-pi-ssh-keys/

alt text

The way to copy the SSH keys in Linux can be achieved using this command

ssh-copy-id -i ~/.ssh/id_rsa IP_ADDRESS

This security settings identifies our main computer and each of the nodes.

This removed the need to use a single password for each node. We make use of the Private and Public SSh using Putty Key Generator

Create SSH keys using PuTTY

On Windows is easy to use Putty Key Generator, and then connect using Putty to each of the nodes, once the Private Key has been shared across all devices alt text

- This key is shared and copied across all devices alt text

Copy Public Keys to each Raspberry pi

The next step for the Keys to work is move to the Raspberry Pi cluster

Run the following set of commands on each Raspberry Pi As we have connected via SSH and VNC you can choose any method to do it so alt text Run this command below for each nodes on the Raspberry Pi cluster

install -d -m 700 ~/.ssh

sudo chmod 644 ~/.ssh/authorized_keys sudo chown pi:pi ~/.ssh/authorized_keys

We establish connection with each node by using the Public Key: alt text

Log in on each node using Private keys

Now we have entered the Raspberry pi by using the public key we have just copied

alt text

Set a static IP for our raspberry pi

In order to access each Raspberry pi outside our local network, we need to assign a static IP to each of them We use static IP to access the Raspberry pi over a long period of time, or when in our project it is been used as a server

Raspbian includes the DHCP client daemon that allows you to change the private address of a computer and set it up as static IP address.

Edit on each pi file dhcpd.conf to provide a statis IP. Firts we specify the type of our internet connection [eth0] ethernet cable or [wlan0] wireless lan One node of my Raspberry pi cluster has IP 192.168.0.14. We define a static Ip by adding in the file below the suffix /24. Also the router always ends in .1

Static IP configuration:sudo nano /etc/dhccpd.conf
Edit dchpc configuration file
interface wlan0 inet dhcp
static ip_address = 192.168.0.14/24
static router = 192.168.0.1**

Save changes, close config file, Restart Raspberry pi: sudo reboot check it works ping raspberrypi.local

Online resource with explanation on how to setup statip IP https://www.ionos.co.uk/digitalguide/server/configuration/provide-raspberry-pi-with-a-static-ip-address

Static IP has been setup

Now we can confirm that the Raspberry Pi as 192.168.0.14 as static IP

alt text

This will allow us to turn it into a server and also to be accessible from outside our local network

alt text

Increase SWAP memory allocation

Then we change the size of the SWQP memory to increase it to 3GB

sudo /bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=3072 sudo /sbin/mkswap /var/swap.1 sudo /sbin/swapon /var/swap.1 sudo sh -c 'echo “/var/swap.1 swap swap defaults 0 0”>>/etc/fstab'

Prevent the unnecessary use of swap memory for protecting your SD card

Add the following line to the sysclt.conf file

sudo nano /etc/sysctl.conf

Setting up the Server -[S02-03]

We will use Nginx to setup our HTML server that will host the Shiny app

Accodring to Andres Castro Socolich, ser can install Nginx following these comands below

sudo apt install nginx

sudo chown -R www-data:pi /var/www/html/

sudo chmod -R 770 /var/www/html/

Application delivery and API management for modern app teams.> https://www.nginx.com/

Create our database using PostgreSQL

We well use again Andres Castro method to install a SQL server on the Raspberry Pi

He recommends PostgreSQL as open source and good features to manipulate our process our data into the Shiny app

Now we install PostgreSQL following these commands sudo apt install postgresql libpq-dev postgresql-client postgresql-client-common

Configure posgresql to accept local and remote connections

We will enable our SQL server to be accessed from inside and also outside our local network

We need to find out the POSGRESQL version we have just installed

alt text

Edit this file with these changes below:

sudo nano /etc/postgresql/13/main/pg_hba.conf

local all all md5 #Modify
host all all 192.168.0.14/24 trust #Add
host all all 0.0.0.0/0 password #Add

alt text

Changes applied to pb_hba.conf

Before the changes

alt text

After: sudo nano /etc/postgresql/13/main/pg_hba.conf

alt text

Changes applied to postgresql.conf

listen_addresses #Comment 'listen_addresses='*' #Add'

After the changes alt text Finally we re-start postgresql

sudo systemctl restart postgresql

Create pi user and database in PostgreSQL server

Create Pi user and datbase in PostgreSQL server

Standard way of creating new user in postgres createuser pi -P –interactive

Install R from source in the Raspberry PI

Following Andres scripts we have successfully installed R on the Raspberry pi alt text

Install Shiny-server from Source on the Raspberry PI

One of the drawbacks of this experiment, installing Shiny-server from source in the Raspberry pi can take up to 17 hours

alt text

CONCLUSION

This was an experiment and the amount of RAM required by the Shiny app requires lot of resources from the Raspberry pi cluster

Performance of the cluster will be slow

It's a good experiment to learn about networks setup and how to deploy servers and install remotely software on the Raspberry pi

My recommendation is to build a Shiny app using a laptop or server or any Cloud Service available

This was an experiment and the amount of RAM required by the Shiny app requires lot of resources from the Raspberry pi cluster

My Github repo with the R and Python scripts and documentation

The Gihub Repo called “Raspbery_pi_Shiny_app” includes all scripts and resources to replicate and follow the different sections presented in the slides

https://github.com/Pablo-source alt text

Thanks

Any questions?
Thank you all for your time
https://nhsrcommunity.com/
https://nhsrcommunity.com/home/webinars/

Resources

Setting your Shiny Server on Raspberry po

https://community.rstudio.com/t/setting-up-your-own-shiny-server-rstudio-server-on-a-raspberry-pi-3b/18982/

https://nhsrcommunity.com/home/webinars/

Tinkernut.How To Make A Cluster Computer (Part 1)

https://www.youtube.com/watch?v=1R0UgIgcb5g&t=4s

Network Chuck. I build a Raspberry Pi super computer
https://youtu.be/X9fSMGkjtug